Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/new modifier implementation #356

Open
wants to merge 35 commits into
base: develop
Choose a base branch
from

Conversation

rfhaque
Copy link
Collaborator

@rfhaque rfhaque commented Aug 27, 2024

New modifier implementation using experiment.py

@github-actions github-actions bot added experiment New or modified experiment ci Involving Project CI & Unit Tests labels Aug 27, 2024
Copy link
Collaborator

@scheibelp scheibelp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a suggestion on refactoring caliper.Caliper into two classes to make it easier to assemble modifier/spec components.

caliper_modifier_modes["mode"] = var
modifier_list.append(caliper_modifier_modes)
# Add time as the last mode
modifier_list.append({"name": "caliper", "mode": "time"})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(question) When you mentioned having to order things yourself, is this what you mean?

In a separate conversation, you mention that this is so "In this case, the time mode is always added last to the modifier list which adds the CALI_CONFIG variable after each of the CALI_CONFIG_MODE variables have been defined. "

I'm not clear on how this achieves that. Is time the only modifier mode that defines CALI_CONFIG?

)
elif self.spec.satisfies("caliper=cuda"):
cuda_support = (
self.spec.satisfies("caliper=cuda") and True
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and True looks vestigial

# set package spack specs
package_specs = {}

if not self.spec.satisfies("caliper=none"):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the impression that some combinations of caliper=... are not allowed. I don't know if we support conflict yet, but FWIW you can raise an exception here for forbidden combinations.

from benchpark.experiment import Experiment


class Caliper(Experiment):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(suggestion, not a problem) IIRC you mentioned that caliper would never be instantiated as an experiment itself. FWIW, this could move from var/exp_repo/experiments/caliper/ to lib/benchpark, possibly into a new subdir like lib/benchpark/experiment_addons

}
}

def compute_modifiers_section(self):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a suggestion for refactoring this to make it easier to add analogs to caliper:

  • Experiment.__init__ (in the base class Experiment) calls Experiment.initialize
  • Move all functionality in caliper.Caliper except for variant declaration into caliper.Helper
  • Amg2023.initialize sets helpers = [caliper.Helper()]
  • Experiment (the base class) handles tacking on extra modifiers from all .helpers
    • It can also take care of caliper_package_specs

The benefit of this organization is that you can add more functionality (that defines other modifiers/specs) without having to modify functions in this class other than .initialize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Involving Project CI & Unit Tests experiment New or modified experiment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants